Automated electric vehicle charging system and method

ABSTRACT

A system and method for charging an electric vehicle includes identifying vehicle information corresponding to the electric vehicle based on an electronic image of the electric vehicle, retrieving from an electronically stored database a location of a charging port on the electric vehicle based on the vehicle information, and robotically moving a charging connector according to the retrieved location to engage the charging port of the electric vehicle to charge a battery.

CROSS-REFERENCE TO RELATED PATENT APPLICATION

This application claims priority to and the benefit of ProvisionalApplication Ser. No. 61/325,643, filed on Apr. 19, 2010, and ProvisionalApplication Ser. No. 61/328,411, filed on Apr. 27, 2010, the disclosureof which is incorporated by reference herein in its entirety.

TECHNICAL FIELD

The present disclosure relates to automatically charging an electricvehicle, and more particularly, to a system and method for automaticallycharging an electric vehicle.

DISCUSSION OF RELATED ART

With the growing availability of electric vehicles, there will be a needfor charging stations to facilitate the operator of an electric vehicleto ‘fill-up’ the electric charge of his vehicle. The operator may haveto manually perform certain actions, including charging the vehicle,entering payment information, selecting options (e.g., fast or slowcharging), or initiating and/or approving the charging operation.Although a charging station at home may be programmed to automaticallycharge a vehicle at night to make use of lower power rates duringoff-peak hours, the operator of a vehicle may wish to manually chargethe vehicle during other times. For example, if a vehicle has been usedfor part of a day, the operator may wish to manually initiate chargingduring the day so that the vehicle is fully charged for use later thatsame day. As a result, the operator of the vehicle may have to handle ahigh-voltage cable or connector, which may be dangerous, especiallyduring inclement weather.

BRIEF SUMMARY

According to an exemplary embodiment of the present disclosure, a methodfor charging an electric vehicle includes identifying vehicleinformation corresponding to the electric vehicle based on an electronicimage of the electric vehicle, retrieving from an electronically storeddatabase a location of a charging port on the electric vehicle based onthe vehicle information, and robotically moving a charging connectoraccording to the retrieved location to engage the charging port of theelectric vehicle to charge a battery.

According to an exemplary embodiment of the present disclosure, a methodfor charging an electric vehicle includes identifying vehicleinformation corresponding to the electric vehicle using aradio-frequency identification (RFID) tag mounted on the vehicle,retrieving from an electronically stored database a location of acharging port on the electric vehicle based on the vehicle information,and robotically moving a charging connector according to the retrievedlocation to engage the charging port of the electric vehicle to charge abattery.

According to an exemplary embodiment of the present disclosure, a systemfor charging an electric vehicle includes a first camera, an electronicdatabase, a processor, and a robotic arm. The first camera is configuredto acquire an electronic image of the electric vehicle. The processor isconfigured to identify vehicle information corresponding to the electricvehicle based on the electronic image of the electric vehicle, andretrieve from the electronic database a location of a charging port onthe electric vehicle based on the vehicle information. The robotic armis configured to move a charging connector according to the retrievedlocation to engage the charging port of the electric vehicle to charge abattery.

According to an exemplary embodiment of the present disclosure, a systemfor charging an electric vehicle includes a radio-frequencyidentification (RFID) reader, an electronic database, a processor, and arobotic arm. The RFID reader is configured to identify vehicleinformation corresponding to the electric vehicle using an RFID tagmounted on the electric vehicle. The processor is configured to retrievefrom the electronic database a location of a charging port on theelectric vehicle based on the vehicle information. The robotic arm isconfigured to move the charging connector according to the retrievedlocation to engage the charging port of the electric vehicle to charge abattery.

According to an exemplary embodiment of the present disclosure, a methodfor charging an electric vehicle includes acquiring a plurality ofimages of a field of view while in a vacant state, detecting whether theelectric vehicle has entered the field of view based on an analysis ofthe plurality of images while in the vacant state, initiating a trackingstate upon detecting that the electric vehicle has entered the field ofview, tracking a position and pose of the electric vehicle while theelectric vehicle is in motion based on a plurality of features of theelectric vehicle extracted from each image while in the tracking state,initiating an identify state upon detecting that the electric vehicle isno longer in motion, identifying a matching vehicle model candidate in adatabase using the position and pose of the electric vehicle in eachimage while in the identify state, initiating a connect state uponidentifying a matching vehicle model candidate, verifying whether acharging port on the electric vehicle is in an expected location basedon a most recently acquired image while in the connect state, andengaging a charging connector into the charging port upon verifying thatthe charging port is in the expected location while in the connectstate.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The above and other features of the present disclosure will become moreapparent by describing in detail exemplary embodiments thereof withreference to the accompanying drawings, in which:

FIG. 1 is a flowchart showing the operation of a vacant state softwaremodule, according to an exemplary embodiment;

FIGS. 2A-2B are a flowchart showing the operation of a tracking statesoftware module, according to an exemplary embodiment;

FIG. 3 is a flowchart showing the operation of an identify statesoftware module, according to an exemplary embodiment;

FIG. 4 is a flowchart showing the operation of a connect state softwaremodule, according to an exemplary embodiment;

FIG. 5 is a flowchart showing the operation of a charging state softwaremodule, according to an exemplary embodiment;

FIG. 6 is a flowchart showing the operation of a disengage statesoftware module, according to an exemplary embodiment;

FIG. 7 is a flowchart showing the operation of a departing statesoftware module, according to an exemplary embodiment;

FIG. 8 is a block diagram of the vehicle charging system, according toan exemplary embodiment; and

FIG. 9 is a computer system for implementing a method of automaticallycharging an electric vehicle, according to an exemplary embodiment ofthe present disclosure.

DETAILED DESCRIPTION

Exemplary embodiments of the present disclosure will be described morefully hereinafter with reference to the accompanying drawings. Likereference numerals may refer to like elements throughout theaccompanying drawings.

System Components

An exemplary embodiment of an automated electrical vehicle chargingsystem may include a processor, a camera and an associated computerinterface board (e.g., a frame grabber), a charging cable and a chargingconnector, and a robotic arm. The electric vehicle charging station mayautomatically connect to a vehicle and charge the vehicle without theneed for operator action. The charging station may operate without theuser having to precisely align the vehicle at the charging station, andwithout any specialized modifications or equipment installations made tothe vehicle. For example, when a vehicle is driven up to and parked nearthe charging station, the charging system may recognize that the vehicleis present, identify information about the vehicle (e.g., the make,model and year of the vehicle), and determine the location of thevehicle's charging port using this information. The charging system maythen automatically establish a connection with the vehicle's chargingport and initiate the charging process. The charging system may utilizedifferent charging voltages and currents, and various types ofelectrical connectors. Once charging is complete, the charging systemmay disengage the connection and monitor the vehicle to determine whenit has left the charging system, allowing another vehicle to be charged.

According to an exemplary embodiment, an automated electrical vehiclecharging system uses a camera to continuously monitor the space where avehicle is to be parked for charging. The camera analyzes the imageframes to identify when a vehicle moves into the camera field of view.When a vehicle is found, a sequence of images is used to track thevehicle until it is parked, and those images are used to identifyinformation about the vehicle such as, for example, the vehicle's make,model and year. A database of three dimensional (3D) mathematical modelsfor different vehicles and their corresponding make, model and year iscompared to the camera images, similarity scores are calculated, and avehicle having the highest similarity score is selected.

Once the make, model and year of the vehicle are determined, thelocation of the vehicle's charging port is obtained from a look-up inthe vehicle database. The location of the charging port may be verifiedfrom the camera image of the parked vehicle, and a robotic arm may carrya charging cable with a charging connector to the vehicle's chargingport, using a real-time sequence of camera images for guidance. Therobotic arm may open a door covering the charging port, if present,before plugging in the charging connector. Once the charging connectoris engaged in the charging port, the charging system initiates andmonitors the charging process. When the charging system senses thatcharging is complete, the robotic arm disengages the charging connectorand returns to its home position. The charging system continues tomonitor and track the vehicle to determine when it has left the chargingstation. Once the vehicle has left the charging station, the cameramonitors the space to determine when another vehicle has arrived.

The processor operates continuously and executes various softwarefunctions and controls the actions performed by the components of thecharging system. The processor may be part of an embedded computer thatoversees the operation of the entire charging system, including theacquisition of the camera images and control of the robotic arm, andperforms the computations for image analysis and vehicle tracking andidentification. The processor executes the software that performs thevarious functions of the charging system. The embedded computer may bebased on an Intel® S5520SC workstation motherboard with an Intel® Xeon®5500 series quad core processor operating at about 3.3 GHz, however, aswill be appreciated by one having ordinary skill in the art, theembedded computer is not limited thereto. Utilization of multipleprocessor cores in the embedded computer allows the image processingfunctions, the robotic arm control, and other functions to beconcurrently active without competing for processor resources (e.g.,when the camera images are being used to guide the charging connectorinto the vehicle charging port). The embedded computer may include avision processor board such as, for example, a Matrox® Odyssey Xpro+scalable vision processor board, which includes a PowerPC® G4 processorwith a customizable Field Programmable Gate Array (FPGA) co-processor.The vision processor board provides highly parallel execution of manyimage operations such as, for example, filtering, motion detection, andmatrix processing. Utilization of the vision processor board takes aportion of the computational load off of the main processors.

The vision processor board may be configured with a Camera Linksingle-camera full-configuration frame grabber mezzanine, which providesthe interface to the camera. The Camera Link frame grabber handles theacquisition of images from the camera, and its interface provides forcontrol of many camera operating parameters such as, for example, theframe rate and shutter speed. The interface can also be used to triggerthe shutter. The shutter speed may be adjusted to compensate for changesin light levels, and to keep the images within an acceptable brightnessrange without being saturated.

The charging system may further include communication devices capable ofsending and receiving messages to other computers or devices. Thecommunication devices may operate via wires or wirelessly, and mayconnect to the Internet using the TCP/IP protocol. The communicationdevices may be used, for example, to download updates to the vehicledatabase, and/or to validate billing information with a remoteaccounting system.

The camera may be used to acquire images of a charging space at thecharging station where the vehicle is to be parked during charging. Theacquired images are used for vehicle tracking and identification, aswell as to guide the robotic arm as it engages the charging connectorinto the charging port. The camera may be, for example, a monochrome,progressive scan charge-couple device (CCD) camera with 1600×1200 squarepixels and the Camera Link interface, such as, for example, a Sony®XCL-U1000. However, as will be appreciated by one having ordinary skillin the art, the camera is not limited thereto. The camera may be ahigh-sensitivity camera (e.g., 400 lux at f/5.6) with shutter speeds ofabout 10 μsec, and may be operated at a frame rate of about 5 Hz toabout 15 Hz, however, the camera is not limited thereto. The cameraacquires images continuously, and the frame grabber places the imagesinto a multi-frame buffer, which allows the software to process the mostrecent image while the next image is being acquired. The camera may bemounted in a fixed position and have a fixed viewing direction, suchthat the side of the vehicle including the charging port is visible whenthe vehicle is parked in the charging space, and the front of thevehicle is visible at some point during the parking process. Amoderately wide angle lens may be used, such that entire chargingstation and the vehicle are within the field of view when the vehicle isparked for charging.

In an exemplary embodiment, an additional camera may be mounted on therobotic arm. The additional camera may acquire images which are used toverify the location of the charging port on the vehicle and guide therobotic arm as it engages the charging connector into the charging port.The additional camera may have a lower resolution than the main cameraused to monitor the vehicle. For example, the additional camera may be amonochrome progressive scan CCD camera with 640×480 square pixels andthe Camera Link interface, such as, for example, a Sony® XCL-V500.However, as will be appreciated by one having ordinary skill in the art,the additional camera is not limited thereto. The additional camera maybe mounted in a fixed position on the robotic arm such that the chargingconnector and the charging port on the vehicle are both within the fieldof view. Utilization of the additional camera allows the vehicle to beparked at the charging station in a position where the charging port isnot visible from the main camera.

The charging cable and the charging connector mate with the vehicle'scharging port and supplies the charging voltage and current to thevehicle. The charging cable and charging connector also carry any signallines associated with the connection. Various types of cables andconnectors may be used to charge the vehicle. The charging connector isattached to the robotic arm, and the robotic arm engages the chargingconnector into the charging port. The charging connector may engage thecharging port via various means. For example, the charging connector maybe pushed straight into the charging port, and an actuator attached tothe robotic arm may be used to twist the charging connector into placeonce it has been pushed into the charging port.

Connectors used in the charging system may be conductively coupled(e.g., directly coupled) or inductively coupled (e.g., magneticallycoupled). When the connectors are conductively coupled, direct contactis made between the conductors, and the supplied voltage may be eitheran AC voltage or a DC voltage. When the supplied voltage is an ACvoltage, a charger/regulator may be located on the charging station oron the vehicle, and may be used to convert the AC voltage to a DCvoltage and to regulate the voltage and/or current as the battery isbeing charged. Disposing the charger/regulator on the vehicle allows thevehicle to be charged at multiple locations. When the supplied voltageis a DC voltage, a charger for voltage and current regulation may belocated on the charging station. When the connectors are inductivelycoupled, the supplied voltage may be an AC voltage, and the conductorsmay be enclosed and impervious to water. In a charging system usinginductively coupled connectors, grid power is first converted to ahigher frequency to improve efficiency and to allow for reasonably sizedconnectors.

The charging cable and charging connector may be compliant with the SAEJ1772 standard, which is a U.S. standard for electrical vehicleconnectors maintained by the Society of Automotive Engineers. Thecharging cable and charging connector may be compliant with variousrevisions of the SAE J1772 standard including, but not limited to olderrevisions (e.g., the Avcon standard) and newer revisions (e.g., theJanuary, 2010 revision). For example, the charging connector may be around connector about 43 mm in diameter and may contain five pins, asdefined in the January, 2010 revision. The five pins are AC Line 1, ACLine 2/Neutral, Ground, Proximity Detection, and Control Pilot.According to the standard, the Proximity Detection line preventsmovement of the car while the charging connector is attached, and the

Control Pilot line facilitates communication for coordinating thecharging level between the vehicle and the charger, as well as otherinformation. The standard further provides safety, particularly in wetweather, by isolating the connection pins on the interior of theconnector with no physical access when mated, and ensuring that there isno voltage on the pins when not mated. The standard defines two charginglevels depending on whether pin 2 is used as an AC line for 240 V orNeutral for 120 V:

AC Level 1: 120 V, single phase, 16 A, (1.9 kW)

AC Level 2: 240 V, single phase, 80 A, (19.2 kW)

An example of a connector based on the January 2010 revision ismanufactured by Yazaki. Other connectors are being developed by Mennekesand are expected to be included under the IEC 62196 electric vehiclestandard. These connectors use three-phase AC and are capable ofcharging at rates used for fast charging.

For purposes of the present disclosure, it is assumed that existingcharging stations monitor any signal wires present in the chargingconnector. For example, signal wires indicating whether a validconnection has been established, the charging level, and other statusinformation may be used to make this information available to thecharging station through a digital input. Similarly, it is assumed thatan existing charging station is turned on or off via a switch or a relaycontrolled by the embedded computer of the present disclosure, and thatany voltage or current issues may be handled by existing equipment.

The robotic arm is used to hold the charging cable and the chargingconnector, and to engage the charging connector into the vehicle'scharging port. The robotic arm may be, for example, a six-axis arm witha reach of about 2 meters and a payload capability of about 10 kg, suchas, for example, a model M-20/A/10L manufactured by FANUC® Robotics.However, as will be appreciated by one having ordinary skill in the art,the robotic arm is not limited thereto. In an exemplary embodiment, anadditional camera used to verify the location of the charging port andguide the robotic arm as it engages the charging connector into thecharging port may be mounted on the robotic arm. Further, an actuatorused to open a cover or door on the charging port, if present, may beattached to the robotic arm.

In an exemplary embodiment, the charging station may include an RFIDreader. RFID is often used for inventory control in retail stores, andis an accepted method for paying by credit card at gas stations andother establishments. The RFID reader may read an RFID tag mounted onthe vehicle to identify information relating to the vehicle such as, forexample, the make, model and year of the vehicle. The RFID reader mayfurther be used to provide customer billing information for the chargingtransaction. The RFID reader may be a commercial embedded RFID readersuch as, for example, a SkyeModule™ M10 manufactured by Skyetek®, whichhas a range of about 5 meters, however, the RFID reader is not limitedthereto. The RFID reader may be used in place of, or in conjunctionwith, the computer vision software techniques described herein toidentify the make, model and year of the vehicle. Exemplary embodimentsmay take the cost of each implementation (e.g., the cost of the RFIDreader compared to the incremental cost of the added computational powerused to perform the described computer vision software techniques) intoconsideration when determining which implementation to utilize. The RFIDreader may further be used to provide billing information for thecharging transaction. When RFID is utilized, a camera may still be usedto track the position and pose of the vehicle, allowing for thedetermination of the location of the vehicle's charging port withrespect to the charging station.

In an exemplary embodiment, an energy storage device may be used toprovide for fast charging of the vehicle at power rates beyond thecapability of the connection to the local power grid to the chargingstation. For example, the energy storage device may be a battery similarto the 26 kWh lithium-ion battery used in the Nissan® Leaf™, however,the energy storage device is not limited thereto. Utilization of anenergy storage device allows the charging system to provide fastcharging without the need for special wiring.

Exemplary embodiments may further include other components that are usedin existing charging stations such as, for example, power cables andconnectors of various types, voltage/current regulators, and credit cardreaders.

Vehicle Tracking and Recognition, and Software Architecture

The process of tracking a vehicle to determine when it has entered thecharging station, and identifying information about the vehicle such as,for example, the make, model and year of the vehicle, is based on astructure-from-motion algorithm, in which a sequence of two-dimensional(2D) camera images of the moving vehicle are analyzed to construct a 3Dstructure, including the pose of the vehicle. An example of astructure-from-motion algorithm is described in Prokaj, J., Medioni, G.,3-D Model Based Vehicle Recognition, IEEE Workshop on Applications ofComputer Vision (2009), and Hartley, R., Zisserman, A., Multiple ViewGeometry in Computer Vision, Cambridge University Press (2003). In anexemplary embodiment, a single camera mounted in a fixed positioncaptures the images as the vehicle is driven into the charging space. Asthe vehicle moves, it appears in a different position and presents adifferent pose to the camera in each image frame. When tracking thevehicle, feature points on the vehicle, which appears as a rigid body,follow different trajectories as a result of the perspective projectionof the 3D object (e.g., the vehicle) onto the camera image plane.Feature points between different frames may be matched and a system ofequations may be generated. Solution of the equations results in a 3Dstructure corresponding to the vehicle.

This analysis is similar to a stereo vision system, where images of astationary object are captured from different viewing angles to providedepth information that is lacking in any one image. In an exemplaryembodiment, the viewpoint and viewing direction of each image withrespect to the vehicle coordinate frame are not known in advance, andare obtained using the solution of the system of equations describedabove. Further, a relatively large number of image frames may be used,and the frames may be acquired at a rate that is sufficient to make thedistance that the vehicle travels between successive frames relativelysmall, and the motion relatively smooth. As a result, the identificationof corresponding features in adjacent frames is improved compared to atraditional stereo vision system. This results in improved reliabilitywhen recognizing whether a feature is no longer visible (e.g., a featurethat is occluded in certain views).

The software used for the charging system may be built on an operatingsystem platform that supports hard real-time processing such as, forexample, Wind River Linux 3.0 with Wind River Real-Time Core for Linuxfrom Wind River Systems, Inc., however, the software is not limitedthereto. The software architecture may be organized as a set of systemstates, and each state may be associated with a processing module thatis executed when the charging system is in that state. The system statesare discussed in detail with reference to FIGS. 1 to 7.

If a module that is currently executing changes the system state, thatmodule is exited, and the module for the new state is executed. Thearchitecture also includes a database containing vehicle informationsuch as, for example, the make, model, and year of different vehicles,which supports the processing modules in identifying the vehicle andlocating its charging port.

The system states and the associated processing modules are describedherein. The states are listed below in the order they are frequentlytraversed. For example, a vehicle may first be driven into the field ofview of the camera, proceed through the charging process, and then leavethe charging space. However, as will be appreciated by one havingordinary skill in the art, the order of the states is not limitedthereto, and the states may be traversed in any order.

FIG. 1 is a flowchart showing the operation of the vacant state softwaremodule, according to an exemplary embodiment.

The vacant state is the initial or idle state of the charging system. Inthe vacant state, no vehicle is at or approaching the charging station.Camera images are obtained and analyzed in the vacant state to detectwhether a vehicle has entered the field of view of the camera. When avehicle is detected, the system state is set to the tracking state.

In the vacant state, when there is no vehicle in the charging space, theimage frames acquired by the camera are used to estimate the backgroundusing the mode of the images over the preceding several seconds. Avehicle coming into the camera field of view is recognized using abackground-subtraction motion detector technique, where the area thatchanges is large enough to represent a vehicle and is completely withinthe field of view. When a vehicle is found, the images are saved untilthere is no motion of the vehicle (e.g., the vehicle is parked). Themotion may be paused and restarted, resulting in additional images beingadded to the sequence until the vehicle stops again.

Referring to FIG. 1, when the system is in the vacant state, no vehiclehas been detected at the charging station, and no vehicle has beendetected approaching the charging station. In the vacant state, eachcamera frame is processed as it is acquired to determine whether amoving vehicle is present. For example, if the system is not currentlymonitoring a moving vehicle (block 101), the next camera frame isobtained (block 102). Background subtraction is used to detect whetheran object is present (blocks 103-105). If an object is detected (block106), the blob of pixels representing the object is monitored. If theobject is completely contained within the field of view (e.g., no partof the object is at edges of the image) (block 107), is large enough tobe a vehicle (block 108), and is moving (e.g., compared to the previousframe) (block 109), the system state is set to the tracking state (block110). If an object is not found (block 106), is not completely containedwithin the field of view (block 107), is not large enough to be avehicle (block 108), or is not moving (block 109), the background isupdated (block 105), and it is determined whether the database should bechecked for updates (block 111). If it is not time to check for updates,the next camera frame is obtained (block 102). If it is time to checkfor updates, a thread is initiated which sends a message via thecommunication link to determine whether updates to the database areavailable (blocks 112, 113). The thread may execute at a low priority ina background mode. If updates are available, they are downloaded andautomatically installed (blocks 114, 115).

FIGS. 2A-2B are a flowchart showing the operation of the tracking statesoftware module, according to an exemplary embodiment.

In the tracking state, the trajectory of the vehicle is tracked fromframe to frame. Features are identified on the vehicle and compared withfeatures from previous frames to establish a track for the features. Thedata for each camera frame is saved for subsequent further processing.Once the vehicle has been parked, the system state is set to theidentify state.

Referring to FIGS. 2A-2B, as each frame is acquired by the camera (block201), pre-processing is performed on the frame to provide preliminaryanalysis of the vehicle motion, and to identify and track the featurepoints used by the structure-from-motion algorithm. Backgroundsubtraction is used to detect whether the vehicle is present (blocks202, 203). If the vehicle is not found, or if the vehicle was previouslyfound but has exited the field of view, the system state is set to thevacant state (block 204). If the vehicle is found, the center of thevehicle in the image is computed (block 205), and the blob of pixelsrepresenting the vehicle is compared with the previous frame todetermine whether the vehicle is still moving (block 206). If thevehicle has not moved for a certain period of time, the vehicle isconsidered to be parked, and the system state is set to the identifystate (blocks 208, 209). If the module has not switched to a new state,the feature matching and tracking processing for the current image frameis performed.

Corner detection may be used to extract features in each frame fortracking the vehicle motion and for identifying the make, model and yearof the vehicle (block 207). For example, a Harris corner detector may beused. An example of a Harris corner detector may be found in Harris, C.,Stephens, M. J., A Combined Corner and Edge Detector, Proceedings, AlveyVision Conference pp. 147-152 (1988). The Harris corner detector isbased on the local autocorrelation matrix, whose eigenvalues represent ameasure of the change in intensity in the two principal directionsdefined by the eigenvectors. If both eigenvalues are small, there islittle change in any direction (e.g., the intensity is nearly constantover that part of the image). If one eigenvalue is large while the otheris small, it indicates an edge perpendicular to the first eigenvector.If both eigenvalues are large, it indicates a corner. The threshold maybe set so that a moderately sparse set of corner features is extracted.Once candidate features are identified using, for example, the Harriscorner detector (block 207), each one is compared for correspondencewith features and tracks from previous frames using a feature matchingtechnique, as shown in FIG. 2B.

If a feature in the current frame matches a previously establishedfeature track, it is added to that track. Since the camera frame rate isselected to keep the apparent motion small from one frame to the next,and because the vehicle is physically constrained by the wheels on theground plane, the vehicle trajectory and any feature tracks are smooth.Features are matched across image frames using, for example, a Bayesianmaximum a posteriori (MAP) technique to find the feature in the currentframe that best matches a feature in a previous frame. Once a match isfound, a track is established, identifying the relationship between thefeatures and their positions in their corresponding frames. Features inthe current frame are also tested against previously established tracksand are added to the track if a match is found. For example, the searchfor matches starts by projecting each track found in previous frames(block 211) to estimate the position where the feature would be locatedin the current frame (block 212). The Bayesian prior probability istaken as a circular distribution centered at that location, decreasingwith distance, from a maximum at the center out to a radius of about 1.5times the apparent motion from the last frame of the track. Thelikelihood function is calculated as a normalized cross-correlationbetween features in the two frames (block 213), and the feature in thecurrent frame with the maximum a posteriori probability (block 214),above a certain threshold (block 216), is selected as the match andadded to the track (block 217). This is done for each track (block 215),and for each feature in the current frame (blocks 210, 218).

If no matching track is found, it is compared to features in previousframes, and if a match is found, a new track is created. The datagenerated for each camera frame is saved for subsequent furtherprocessing. That is, after all previous tracks have been examined, anyunmatched features from the previous few frames are examined formatching with features in the current frame (block 219). The process issimilar to the process used for tracks, except that the center of theprior distribution is estimated from the feature location in theprevious frame projected by the motion of the center of the blob ofvehicle pixels from the previous frame to the current frame (blocks 220,221). The feature in the current frame with the maximum a posterioriprobability (block 222), above a certain threshold (block 224), isselected as the match, and a new track is established (block 225) bydoing this for each feature (block 223). Any features in the currentframe that are not used in a track remain unmatched.

FIG. 3 is a flowchart showing the operation of the identify statesoftware module, according to an exemplary embodiment.

In the identify state, the feature matching/tracking results saved foreach frame in a structure-from-motion algorithm are applied to determinethe position and pose of the vehicle in each frame. The vehicle databaseis then searched to find the best match of pose-corrected features inthe database with the frame data. Once the best match is determined, thesystem state is set to the connect state.

Once the structure-from-motion algorithm obtains the position and poseof the vehicle as seen in each of the camera frames, the position andpose are used to identify the make, model and year of the vehicle. Theposition and pose of the vehicle are used to match the featureinformation extracted in each frame with the 3D models in the vehicledatabase. Each 3D vehicle model may be translated and rotated to matchthe pose of the vehicle in the frame, converted to a 2D image, andmatched with the 2D features seen in the frame. As a result, the searchmay be performed in 2D rather than 3D, which may reduce the searchrequirements.

Referring to FIG. 3, a motion detector algorithm is used to test eachframe as it is acquired by the camera determine whether the vehicle ismoving again (blocks 301, 302). If the vehicle is moving, the systemstate is set to the tracking state (block 304), where the systemcontinues to track the vehicle and its feature points, and the identifystate is exited (block 305).

If the vehicle is not moving, the structure-from-motion algorithm isused to determine the position and pose of the vehicle for each framecollected in the tracking state. For example, once featurecorrespondence between frames has been established, a starting frame isselected based on the number and quality of matching features with otherframes (block 306). A second frame, moving forward in the sequence, isthen chosen (block 307), and those features that match the first frameare used to generate a system of linear equations (block 308) which maybe used to obtain the change in position and pose of the vehiclerelative to the first frame (block 309). The second frame is chosen bothfor its matching features with the first frame, and for the apparentmotion of those features from the first frame (block 310). While thecamera frame rate is selected to keep the apparent motion small from oneframe to the next for accurate feature tracking, the use of frames withsmall apparent motion may enhance noise in the solution of theequations. For motion reconstruction, the frame is selected so that theapparent motion produces a well formed set of equations that can besolved with minimal noise. The process repeats incrementally, choosing anext frame and computing the vehicle position and pose, until the end ofthe frame sequence is reached (blocks 307-310). The process is againrepeated, working back from the starting frame to the beginning of theframe sequence. At this point, the position and pose of the vehicle ineach computed frame are known with respect to the starting frame. Theposition and pose are related to the charging station coordinate system,or the camera coordinate system, by modeling the vehicle to have fixedrear wheels with rotation around the vertical axis at the center of therear axel and fitting the vehicle trajectory through the computedposition and pose data points.

The change in position and pose from one frame to the next is calculatedby solving for F, the fundamental matrix of the epipolar geometryrelating the two frames [see, for example, Hartley, R., Zisserman, A.,Multiple View Geometry in Computer Vision, Cambridge University Press(2003)]. F is a 3×3 matrix of rank 2 that can be constructed from eightparameters, including the translation and rotation of the vehiclebetween the frames. The equations are generated using the equationx′^(T) F x=0, where:

-   -   x is a 3×1 matrix containing the coordinates of a feature point        in the image plane of the first frame, and    -   x′^(T) is the transpose of the 3×1 matrix containing the        coordinates of the corresponding feature point in the second        frame.        Each pair of matching feature points produces one equation to        solve for the eight variables, with one additional constraint        that the vehicle motion is confined to the ground plane. Each        pair of frames has at least seven matching features. The        over-constrained problem is solved in a straight forward manner        using, for example, Singular Value Decomposition (SVD). Once a        solution is found, the feature correspondences are checked to        determine that they are on or near their epipolar lines [see,        for example, Hartley, R., Zisserman, A., Multiple View Geometry        in Computer Vision, Cambridge University Press (2003)]. The        check for outliers may be done on a statistical basis using, for        example, the RANSAC algorithm [see, for example, Fischler, M.,        Bolles, R., RANdom SAmpling Consensus: a Paradigm for Model        Fitting with Application to Image Analysis and Automated        Cartography, Commun. Assoc. Comp. Mach., 24:381-395 (1981)]        (block 311). Outliers are removed and the solution is used to        guide the search for additional correspondences (block 312). The        computations used to determine the vehicle position and pose are        repeated.

When it is determined that there are no more significant changes (block313), the vehicle database is searched for a model of a vehicle havingthe best matching make, model and year. In performing the search, asimilarity figure of merit is accumulated over all camera frames used inthe structure-from-motion algorithm. For each frame (block 314), each 3Dmodel in the database is rotated to the vehicle pose for that frame, andthe 3D model features are projected onto the 2D plane corresponding tothe camera image plane (block 316). The similarity factors are thencomputed in 2D (block 317). This process is repeated for each frame usedin the structure-from-motion algorithm (blocks 314, 319), and for eachmake, model and year in the database (blocks 315, 318). When finished,the system state is set to connect state (block 322). If no valid matchis found, the vehicle operator is alerted to enter the make, model andyear manually (e.g., via a keypad) (block 321).

FIG. 4 is a flowchart showing the operation of the connect statesoftware module, according to an exemplary embodiment.

In the connect state, it is first verified whether the vehicle chargingport is in its expected location and is accessible. Camera images arethen used to direct the robotic arm towards the charging port. Chargingprotocols, including, for example, a fast charge protocol, are verified.Once completed, the system state is set to the charging state.

Referring to FIG. 4, the location of the vehicle charging port isobtained from the vehicle database, along with information about thedirection from which the charging connector is to be inserted (block401). The latest camera frame is analyzed to validate the location ofthe charging port and to verify that the charging port is within therange of the robotic arm carrying the charging connector (block 402).Since the position and pose of the charging port are known to a goodapproximation, and since the 3D model can be rotated, translated andscaled to match the pose and position of the charging port, the searchspace is small, and correlation may be used to identify the chargingport. A path is then calculated for the robotic arm to insert thecharging connector into the charging port, providing for properdirection as the charging connector engages the charging port (block403). If a cover or door covers the charging port, the robotic arm mayopen the cover or door. The robotic arm is then directed to move alongthe calculated path. While moving along the path, camera frames areanalyzed to accurately determine the relative position of the chargingport and the charging connector, and to make adjustments to the positionof the charging connector on the robotic arm as it approaches thecharging port (blocks 405-407). The camera frames may be analyzed, forexample, using a correlation technique as described above. Once thecharging connector has been inserted into the charging port, the systemstate is set to the charging state (block 408). In an exemplaryembodiment, an additional camera may be mounted on the robotic arm toguide the charging connector into the charging port. Steps similar tothose described in reference to FIG. 4 may be used in conjunction withthe additional camera. In this embodiment, the charging connectorappears stationary in the camera image and the charging port on thevehicle appears to move.

FIG. 5 is a flowchart showing the operation of the charging statesoftware module, according to an exemplary embodiment.

In the charging state, the charging process is initiated and monitored.Once the charging process is complete, charging voltage/current isturned off, and the system state is set to the disengage state.

Referring to FIG. 5, the charging system outputs a binary output suchas, for example, a DC signal voltage or a relay control signal to theexisting charging station equipment. The binary output indicates thatthe charging connector has been inserted into the charging port and thatthe charging voltage/current may be turned on (block 501). Themonitoring of signal wires in the charging connection may be performedby the existing equipment, or by components integrated into the chargingsystem. For example, when the monitoring of signal wires is performed byexisting equipment, digital inputs in the existing equipment may becontinuously monitored, and the binary output may be turned off when theexisting equipment signals that charging has been completed (blocks 502,503). The system state is then set to the disengage state (block 504).

FIG. 6 is a flowchart showing the operation of the disengage statesoftware module, according to an exemplary embodiment.

In the disengage state, the charging connector is disengaged from thevehicle's charging port, and the robotic arm is returned to its homeposition. Once completed, the system state is set to the departingstate.

Referring to FIG. 6, a path is calculated for the robotic arm todisconnect the charging connector from the charging port and return toits home position (block 601). If a cover or door covers the chargingport, the robotic arm may close the cover or door. The robotic arm androbotic actuator used to close the cover or door, if present, is thendirected to move along the calculated path and return to its homeposition (blocks 602-604). When finished, the system state is set to thedeparting state (block 605).

FIG. 7 is a flowchart showing the operation of the departing statesoftware module, according to an exemplary embodiment.

In the departing state, the vehicle is tracked as it moves away from thecharging station. Once the vehicle has left the charging station, thesystem state is set to the vacant state.

Referring to FIG. 7, a camera frame is obtained (block 701). The pixelscorresponding to the vehicle are obtained based on the known position ofthe vehicle (block 702). The next camera frame is then obtained (703),and the charging system continues to track the vehicle trajectory usinga cross-correlation of the vehicle image from one frame to the next(blocks 704,705). The estimation of the position of the vehicle in thesecond frame results in an efficient search. It is then determinedwhether the vehicle is leaving the field of view (block 706). If thevehicle is not leaving the field of view, the tracking of the vehiclecontinues (blocks 703-705). If the vehicle is leaving the field of view,the system state is set to the vacant state (block 707).

In an exemplary embodiment, a database including information aboutvarious vehicles may be maintained and used with the charging system.The information may be stored, for example, based on the make, model andyear of each vehicle, and may include a 3D model of the vehicle andinformation about the vehicle's charging port. The 3D modelmathematically describes the shape of the vehicle and the location andcharacteristics of key features that may be used to identify the make,model and year of the vehicle. For example, key features may include,but are not limited to, windows, doors, lights, handles and/or bumperson the vehicle. The 3D models may be represented using a vehicle-basedcoordinate system, which may be translated to coordinates based on thecharging station once the position and pose of the vehicle aredetermined. Information stored in the database relating to the vehicle'scharging port may include, but is not limited to, the position and poseof the charging port on the vehicle, as well as the type of coupling,the connector type, the direction from which the connector is inserted,the presence of a cover or door and how it is opened, the chargingvoltages and currents for normal and fast charging, and any connectionprotocols and/or signal wires that apply to the vehicle. The databasemay be created and maintained on a remote computer. The data in thedatabase may initially be preloaded in the charging system, and thecharging system may periodically check for updates via the communicationlink and automatically download and install the updates.

FIG. 8 is a block diagram of the vehicle charging system, according toan exemplary embodiment.

Referring to FIG. 8, an exemplary embodiment of the vehicle chargingsystem 800 includes a processor 802, a first camera 802, an RFID reader803, a robotic arm 804 including a charging connector 805, a secondcamera 806 and an actuator 807, and a communication link 808 connectedto a data bus 809. The processor 801 may be used to implement thecomputer vision software techniques described above. The first camera802 may acquire a plurality of images of a field of view. The RFIDreader 803 may receive customer billing information from an RFID tagmounted on the vehicle and transmit the customer billing information toa remote billing system. The robotic arm 804 may automatically engagethe charging connector 805 into a charging port of a vehicle. Theactuator 807 disposed on the robotic arm 804 may open a door coveringthe charging port of the vehicle prior to engaging the chargingconnector 805 into the charging port, and close the door upondisengaging the charging connector 805 from the charging port. Thesecond camera 806 disposed on the robotic arm 804 may acquire anadditional plurality of images having a viewpoint different from theplurality of images of the field of view. The additional plurality ofimages includes a view of the charging connector 805 and the chargingport. The communication link 808 may check for and download updates to adatabase of the vehicle charging system 800.

Referring to FIG. 9, according to an exemplary embodiment of the presentdisclosure, a computer system 901 for automatically charging an electricvehicle can comprise, inter alia, a central processing unit (CPU) 902, amemory 903 and an input/output (I/O) interface 904. The computer system901 is generally coupled through the I/O interface 904 to a display 905and various input devices 906 such as a mouse and keyboard. The supportcircuits can include circuits such as cache, power supplies, clockcircuits, and a communications bus. The memory 903 can include randomaccess memory (RAM), read only memory (ROM), disk drive, tape drive,etc., or a combination thereof. Exemplary embodiments of presentdisclosure may be implemented as a routine 907 stored in memory 903(e.g., a non-transitory computer-readable storage medium) and executedby the CPU 902 to process the signal from the signal source 908.

As such, the computer system 901 is a general-purpose computer systemthat becomes a specific purpose computer system when executing theroutine 907 of the present disclosure.

The computer platform 901 also includes an operating system andmicro-instruction code. The various processes and functions describedherein may either be part of the micro-instruction code or part of theapplication program (or a combination thereof) which is executed via theoperating system. In addition, various other peripheral devices may beconnected to the computer platform such as an additional data storagedevice and a printing device.

Having described exemplary embodiments for an automated electricalvehicle charging system and method, it is noted that modifications andvariations can be made by persons skilled in the art in light of theabove teachings. It is therefore to be understood that changes may bemade in exemplary embodiments of the disclosure, which are within thescope and spirit of the disclosure as defined by the appended claims.Having thus described exemplary embodiments of the disclosure with thedetails and particularity required by the patent laws, what is claimedand desired protected by Letters Patent is set forth in the appendedclaims.

1. A method for charging an electric vehicle, comprising: identifyingvehicle information corresponding to the electric vehicle based on anelectronic image of the electric vehicle; retrieving from anelectronically stored database a location of a charging port on theelectric vehicle based on the vehicle information; and roboticallymoving a charging connector according to the retrieved location toengage the charging port of the electric vehicle to charge a battery. 2.The method of claim 1, further comprising: acquiring a plurality oftwo-dimensional (2D) images of a field of view; determining whether theelectric vehicle has entered the field of view based on an analysis ofthe plurality of images; extracting a plurality of features of theelectric vehicle from each image; and tracking a position and pose ofthe electric vehicle while the electric vehicle is in motion using theextracted features.
 3. The method of claim 2, further comprising:constructing a three-dimensional (3D) model of the electric vehicleusing the plurality of 2D images, wherein identifying the vehicleinformation is based on a comparison of features in the 3D model of theelectric vehicle with corresponding features in a plurality of 3Dvehicle model candidates stored in the database.
 4. The method of claim2, further comprising: rotating each of a plurality of three-dimensional(3D) vehicle model candidates stored in the database to match a pose ofthe electric vehicle in each of the plurality of 2D images; andconverting each rotated 3D vehicle model candidate to a 2D vehicle modelcandidate, wherein identifying the vehicle information is based on acomparison of features in each of the 2D plurality of images withcorresponding features in each 2D vehicle model candidate.
 5. The methodof claim 2, wherein the position and pose of the electric vehicle aredifferent in each of the plurality of 2D images, and the position andpose of the electric vehicle in each image are used to identify thevehicle information.
 6. The method of claim 2, further comprising:acquiring an additional plurality of 2D images, wherein the additionalplurality of 2D images have a viewpoint different from the plurality of2D images of the field of view, and include the charging connector andthe charging port; verifying the location of the charging port on theelectric vehicle using the additional plurality of 2D images; andguiding the charging connector towards the charging port using theadditional plurality of 2D images.
 7. The method of claim 2, furthercomprising: using background-subtraction to determine whether theelectric vehicle has entered the field of view.
 8. The method of claim1, wherein identifying vehicle information comprises comparing theelectronic image of the electric vehicle with a plurality of vehiclemodel candidates stored in the database.
 9. The method of claim 8,further comprising: calculating a similarity score for each of thevehicle model candidates stored in the vehicle database; and selecting avehicle model candidate having a highest similarity score, wherein thevehicle information is identified using the selected vehicle modelcandidate.
 10. The method of claim 1, wherein the identified vehicleinformation includes a make of the electric vehicle, a model of theelectric vehicle, and a year of the electric vehicle, and the locationof the charging port on the electric vehicle is identified using avehicle model candidate stored in the database having a same make, modeland year as the electric vehicle.
 11. The method of claim 1, furthercomprising: robotically disengaging the charging connector from thecharging port on the electric vehicle.
 12. The method of claim 1,further comprising: robotically removing a door covering the chargingport on the electric vehicle prior to engaging the charging connectorinto the charging port; and robotically covering the charging port withthe door subsequent to disengaging the charging connector from thecharging port.
 13. The method of claim 1, further comprising: receivingcustomer billing information using a radio-frequency identification(RFID) tag mounted on the electric vehicle; and transmitting thecustomer billing information to a remote billing system.
 14. The methodof claim 1, further comprising: downloading an update to the databasefrom a remote system upon determining that the update is available. 15.A method for charging an electric vehicle, comprising: identifyingvehicle information corresponding to the electric vehicle using aradio-frequency identification (RFID) tag on the electric vehicle;retrieving from an electronically stored database a location of acharging port on the electric vehicle based on the vehicle information;and robotically moving a charging connector according to the retrievedlocation to engage the charging port of the electric vehicle to charge abattery.
 16. The method of claim 15, further comprising: acquiring aplurality of images of a field of view; determining whether the electricvehicle has entered the field of view based on an analysis of theplurality of images; extracting a plurality of features of the electricvehicle from each image; and tracking a position and pose of theelectric vehicle while the electric vehicle is in motion using theextracted features.
 17. The method of claim 16, further comprising:acquiring an additional plurality of images, wherein the additionalplurality of images have a viewpoint different from the plurality ofimages of the field of view, and include the charging connector and thecharging port; verifying the location of the charging port on theelectric vehicle using the additional plurality of images; and guidingthe charging connector towards the charging port using the additionalplurality of images.
 18. The method of claim 15, wherein the identifiedvehicle information includes a make of the electric vehicle, a model ofthe electric vehicle, and a year of the electric vehicle, and thelocation of the charging port on the electric vehicle is identifiedusing a vehicle model candidate stored in the database having a samemake, model and year as the electric vehicle.
 19. The method of claim15, further comprising: robotically disengaging the charging connectorfrom the charging port on the electric vehicle.
 20. The method of claim15, further comprising: robotically removing a door covering thecharging port on the electric vehicle prior to engaging the chargingconnector into the charging port; and robotically covering the chargingport with the door subsequent to disengaging the charging connector fromthe charging port.
 21. A system for charging an electric vehicle,comprising: a first camera, configured to acquire an electronic image ofthe electric vehicle; an electronic database; a processor, configured toidentify vehicle information corresponding to the electric vehicle basedon the electronic image of the electric vehicle, and retrieve from theelectronic database a location of a charging port on the electricvehicle based on the vehicle information; and a robotic arm, configuredto move a charging connector according to the retrieved location toengage the charging port of the electric vehicle to charge a battery.22. The system of claim 21, further comprising: an actuator disposed onthe robotic arm, configured to open a door covering the charging port ofthe electric vehicle prior to engaging the charging connector into thecharging port, and close the door covering the charging port upondisengaging the charging connector from the charging port.
 23. Thesystem of claim 21, wherein: the first camera is further configured toacquire a plurality of images of a field of view; and the processor isfurther configured to determine whether the electric vehicle has enteredthe field of view based on an analysis of the plurality of images,extract a plurality of features of the electric vehicle from each image,and track a position and pose of the electric vehicle while the electricvehicle is in motion using the extracted features.
 24. The system ofclaim 23, further comprising: a second camera disposed on the roboticarm, configured to acquire an additional plurality of images, whereinthe additional plurality of images have a viewpoint different from theplurality of images of the field of view and include the chargingconnector and the charging port, and are used to verify the location ofthe charging port on the electric vehicle and guide the chargingconnector towards the charging port.
 25. The system of claim 21, furthercomprising: a communication link configured to download an update to theelectronic database from a remote system upon determining that theupdate is available.
 26. A system for charging an electric vehicle,comprising: a radio-frequency identification (RFID) reader, configuredto identify vehicle information corresponding to the electric vehicleusing an RFID tag mounted on the electric vehicle; an electronicdatabase; a processor, configured to retrieve from the electronicdatabase a location of a charging port on the electric vehicle based onthe vehicle information; and a robotic arm, configured to move acharging connector according to the retrieved location to engage thecharging port of the electric vehicle to charge a battery.
 27. Thesystem of claim 26, further comprising: an actuator disposed on therobotic arm, configured to open a door covering the charging port of theelectric vehicle prior to engaging the charging connector into thecharging port, and close the door covering the charging port upondisengaging the charging connector from the charging port.
 28. Thesystem of claim 26, further comprising: a camera configured to acquire aplurality of images of a field of view, wherein the processor is furtherconfigured to determine whether the electric vehicle has entered thefield of view based on an analysis of the plurality of images, extract aplurality of features of the electric vehicle from each image, and tracka position and pose of the electric vehicle while the electric vehicleis in motion using the extracted features.
 29. The system of claim 26,further comprising: a camera disposed on the robotic arm, configured toacquire a plurality of images, wherein the plurality of images includethe charging connector and the charging port, and are used to verify thelocation of the charging port on the electric vehicle and guide thecharging connector towards the charging port.
 30. The system of claim26, further comprising: a communication link configured to download anupdate to the electronic database from a remote system upon determiningthat the update is available.
 31. The system of claim 26, wherein theRFID reader is configured to receive customer billing information fromthe RFID tag mounted on the electric vehicle and transmit the customerbilling information to a remote billing system.
 32. A computer-readablestorage medium embodying instructions executed by a processor to performa method for charging an electric vehicle, comprising: acquiring aplurality of images of a field of view while in a vacant state;detecting whether the electric vehicle has entered the field of viewbased on an analysis of the plurality of images while in the vacantstate; initiating a tracking state upon detecting that the electricvehicle has entered the field of view; tracking a position and pose ofthe electric vehicle while the electric vehicle is in motion based on aplurality of features of the electric vehicle extracted from each imagewhile in the tracking state; initiating an identify state upon detectingthat the electric vehicle is no longer in motion; identifying a matchingvehicle model candidate in a database using the position and pose of theelectric vehicle in each image while in the identify state; initiating aconnect state upon identifying a matching vehicle model candidate;verifying whether a charging port on the electric vehicle is in anexpected location based on a most recently acquired image while in theconnect state; and engaging a charging connector into the charging portupon verifying that the charging port is in the expected location whilein the connect state.
 33. The computer-readable storage medium of claim32, further comprising: initiating a charging state upon engaging thecharging connector into the charging port; charging a battery in theelectric vehicle while in the charging state; monitoring the charging ofthe battery while in the charging state; initiating a disengage stateupon completion of charging the battery; and disengaging the chargingconnector from the charging port while in the disengage state.
 34. Thecomputer-readable storage medium of claim 33, further comprising:initiating a departing state upon disengaging the charging connectorfrom the charging port; tracking a location of the electric vehicleuntil the electric vehicle is no longer in the field of view while inthe departing state; and initiating the vacant state upon determiningthat the electric vehicle is no longer in the field of view.
 35. Thecomputer-readable storage medium of claim 32, further comprising:establishing a track for at least one of the plurality of features basedon a comparison of the at least one feature in a current image with acorresponding feature in a previous image.